This invention relates to the field of digital image processing method and apparatus technologies. More particularly, the invention relates to image processing methods and image processing apparatus which can apply this image processing methods for typical use with digital photoprinters that read film images photoelectrically to produce prints (photographs) reproducing the images and which are capable of achieving one of the following results: producing a high-quality image even if the input image is taken with low-performance lenses as in films with lens, inexpensive compact cameras and low-cost digital cameras; correcting aberrations such as chromatic aberration of magnification and distortion that develop in the images taken with those low-performance lenses; producing a high-quality image free from the image deterioration due to lens aberrations even if the input image is taken with those low-performance lenses; and particularly performing image processing on the image data representing the image recorded on an image recording medium.
Heretofore, the images recorded on photographic films such as negatives and reversals (which are hereunder referred to simply as xe2x80x9cfilmsxe2x80x9d) have been commonly printed on light-sensitive materials (photographic paper) by means of direct (analog) exposure in which the film image is optically projected onto the light-sensitive material to achieve its areal exposure.
A printer which relies upon digital exposure has recently been commercialized as a digital photoprinter, in which the image recorded on a film is read photoelectrically, converted to digital signals and subjected to various image processing operations to produce image data for recording purposes; recording light that has been modulated in accordance with the image data is used to scan and expose a light-sensitive material to record a latent image, which is subsequently developed to produce a finished print (photograph).
In the digital photoprinter, images are handled as digital image data and the exposing conditions for printing can be determined by image (data) processing. Hence, processing operations such as the correction of washed-out highlights or flat (dull) shadows due to the taking of pictures with rear light or an electronic flash and sharpness enhancement (sometimes referred to simply as xe2x80x9csharpeningxe2x80x9d) can be effectively performed to produce prints of the high quality that has not been attainable by the conventional direct exposure technique. In addition, the synthesizing of images and characters can be accomplished by image (data) processing and, as a ,result, prints can be output after editing and/or processing operations have been performed freely in accordance with specific uses.
Aside from the images recorded on films, the digital photoprinter can also output prints of images recorded with digital cameras or processed with computers. Other than being output as prints, images can be supplied to computers and so forth or stored in recording media such as floppy disks; hence, the image data can be put to various non-photographic uses.
Having these features, the digital photoprinter is basically composed of the following units: a scanner (image reading apparatus) that illuminates the film with reading light and captures the projected light to read the image on the film photoelectrically; an image processing apparatus that performs specified image processing on the image data captured with the scanner or the image data supplied from a digital camera or the like, thereby producing image data for image recording and exposing conditions; a printer (image recording apparatus) that scans and exposures a light-sensitive material to record a latent image on it, for example, by scanning with optical beams in accordance with the image data output from the image processing apparatus; and a processor (developing apparatus) that performs development processing on the printer exposed light-sensitive material to produce a print reproducing the input image.
Users in general who intend to take ordinary pictures seldom use expensive, high-performance cameras such as a single-lens reflex camera but they normally use so-called xe2x80x9ccompact camerasxe2x80x9d that are inexpensive and which are capable of automatic exposing and focusing. Most recently, there are a lot of users who prefer using so-called xe2x80x9cfilms with lensxe2x80x9d by the reason of easy handling.
In cameras such as a single-lens reflex camera that needs a cost to some extent, high-precision lens elements are used, and a plurality of lens elements are combined to record images of very high quality.
In contrast, films with lens and inexpensive compact cameras cannot afford the use of costly lenses and only one or two lens elements may be adopted. With such lens design, images of adequate quality cannot be taken and the image reproduced on prints does not necessarily have high quality.
If the image recorded on films is deteriorated in quality, there is a case that the quality of the output image on prints cannot be adequately improved by the aforementioned corrections. Major causes of the deterioration of the image reproduced from films to be output on prints are lens aberrations such as xe2x80x9cchromatic aberration of magnificationxe2x80x9d and xe2x80x9cdistortionxe2x80x9d that, originate from the low performance of the lenses mounted in the camera used to take the input image.
Color images are formed of three primary colors, for example, red (R), green (G) and blue (B). The refractive index (imaging magnification) of a lens, even if it is a single element, varies subtly with wavelength and differing refractive indices occur with R, G and B lights. In other words, even though the same position in a particular scene, a focused position on a film are slipped off and differ among the R, G and B lights. This is the phenomenon generally called xe2x80x9cchromatic aberration of magnificationxe2x80x9d and the image reproduced from the film has a definite color divergence.
In order to obtain a satisfactory and appropriately recorded image, a plane of a scene of interest that is perpendicular to the optical axis must be focused on the same plane as the imaging plane perpendicular to the optical axis. In fact however, ordinary lenses have the imaging plane displaced along the optical axis and the resulting displacement of the focused position in the axial direction causes a distortion of the focused object. As a natural consequence, the reproduction of the image on the film is distorted.
Other causes of the image deterioration are the reduction of the brightness at the edge of image field which means a phenomenon in which the peripheral area of the image looks darker than the central area which is closer to the optical axis corresponding to the performance of the lens used, and the point spread function (PSF) which is attributable to differing focal positions in the plane of the film.
As noted above, if one uses a camera such as a single-lens reflex camera that needs a cost to some extent, high-precision lens elements may be used and a plurality of lens elements combined to correct various aberrations including chromatic aberration of magnification, distortion, deterioration of marginal lumination and PSF and an appropriate image can be recorded on the film.
However, cameras such as films with lens and compact cameras required to be a low cost can not use high-cost lenses and aberrations will develop in the images recorded on films. As a result, the images reproduced on prints will eventually have color divergence and distortion.
To deal with this problem of image deterioration involving the difficulty in improving the quality of output images on prints, techniques have been proposed in connection with image processing methods and apparatus that correct image aberrations in accordance with the characteristics of lens aberrations that are obtained via certain image acquisition means and two typical examples of such technology are disclosed in Unexamined Published Japanese Patent Application (kokai) Nos. 311425/1994 and 281613/1997, the latter being assigned to the present Applicant. According to these patents, the proposed technology can correct aberrations due to lenses and prevent the deterioration of image quality in the marginal area, thereby ensuring the production of high-quality images at all times.
Specifically, Unexamined Published Japanese Patent Application (kokai) No. 281613/1997 proposes a process of correcting the problem of deterioration of marginal lumination in a photographic processing apparatus and method. In the process, the quantity of light f(i,j) on an image in a given pixel position (i,j) is multiplied by a correction coefficient g(i,j) based on lens characteristics and the obtained product fxe2x80x2 (i,j) is substituted as the corrected quantity of light in the pixel position (i,j). To make the correction for the entire part of the image, j is first moved with i held constant and then i is moved, or alternatively, i is first moved with j held constant and then j is moved; in either way, the whole image can be corrected. When correcting distortion and chromatic aberration of magnification by this method, the position of the subject in an image of interest is changed so that a huge frame memory is required to store the information about all pixel positions of not only before correction but also after the correction. In addition, a circuit, is necessary that performs two processing schemes for i and j as described above and this not only increases the cost of the apparatus but also causes a significant drop in the correction speed, thus there is a problem the practical use of the apparatus is difficult.
On the other hand, unexamined Published Japanese Patent Application (kokai) No. 311425/1994 discloses an image correcting apparatus capable of rapid image correction in accordance with the characteristics of lens aberrations. In this apparatus, the subjects of the correction are the amount of defocusing, the decrease in the quantity of light, and the degree of unsharpness in the hue and chromas of a color image. According to the disclosure, a quantity of deterioration in each of these correction subjects increases as it goes from the center of the image to the peripheral area so that the data of specified patterns that are increased as going from the center toward the peripheral in an image area that is formed by concentric circles or squares extending radially outward from the center of the image are only used for each of given lens characteristics as correction enhancement coefficients for correcting these correction subjects. This approach is capable of rough correction but not image corrections aberrant from the patterns that is prepared beforehand. Hence, there is a problem that it is impossible to perform appropriate corrections according to the characteristics of individual taking lenses.
If one wants to accomplish the appropriate correction by this technology, correction enhancement coefficients must be made available for all patterns that are predicted for the given lens characteristics and to meet this need, a memory of sufficiently large capacity is required. What is more, if the available patterns are not simple concentric circles or squares whose center coincides with that of the image, an increased amount of data has to be calculated to correct matrices and the overall image processing speed is lowered. These practical problems with cost and processing speed are particularly serious in the case of print services that involve the volume reproduction of images.
As mentioned hereinabove, an image processing system is conventionally known that is able to perform various image processing schemes on the image data obtained by reading the film image recorded on photographic films or on the image data input from a digital camera or the like and which then outputs an image in various output modes such as by recording the image on recording materials such as photographic paper or storing the image data in information recording media. Compared to a conventional photographic processing system that records the film image by areal exposure, the image processing system just described above can control the output image quality by image processing onto the image data, thereby output images of high quality are realized.
Speaking of films with lens, the lens is usually composed of an inexpensive plastic lens that inevitably suffers from great amounts of aberrations such as distortion and chromatic aberration of magnification. Hence, the film image recorded on a photographic film by exposure with the film with lens has a comparatively high level of geometric distortion according to the lens distortion (so-called xe2x80x9cpincushion-type distortions) as typically shown in FIG. 25A (FIGS. 25A and 25B illustrate how an image consisting of multiple lines in a grid pattern appears if it is recorded on a photographic film by shooting with a film with lens); at the same time, a color divergence occurs at a comparatively high level due to chromatic aberration of magnification. To deal with this problem, distortion correction for correcting the geometric distortion of an image due to the distortion of the lens on the film with lens and magnification chromatic aberration correction for correcting the color divergence in an image due to the chromatic aberration of magnification of the same lens are being review in order to ensure that the image processing system described above can produce an output image of high quality from the original (input) image recorded with the film with lens.
With a view to increasing the speed of the various processing schemes to be performed with the above-mentioned image processing system, the contents of image processing and the value of a parameter to be applied in a specific image processing scheme have been set identical in specified directions, such as vertical and horizontal, of the image represented by the image data. For instance, if the image processing to be done is electronic scaling, the electronic scaling ratio is set identical in both the vertical and horizontal directions of the image represented by the image data to be processed.
What is unique about image data is that its volume is tremendous and that it represents an image having la two-dimensional extent. Hence, image processing schemes such as ones for correcting distortion and chromatic aberration of magnification are so much complicated in contents that they not only take prolonged time but also require storage means of large capacity. As a result, the image processing section capable of performing image processing schemes such as the correction of distortion and chromatic aberration of magnification is considerably complicated in configuration and, what is more, the processing performance of the image processing system is eventually deteriorated.
Consider, for example, the correction of distortion. First, distortion correcting data representing the direction and the amount of the shift in the position of each of the pixels in the original film image due to the distortion of a lens, as referenced to the inherent position of each pixel (the position of its grid point) composing the film image, are preliminarily measured and stored for each of the lens types used; given the image data to be processed, the distortion correcting data associated with the lens type used in actual shooting are captured; on the basis of the captured distortion correcting data, the positions of the pixels represented by their data in the case where no distortion is present are evaluated; and the density value at the inherent position of a particular pixel (the position of its grid point) is determined by arithmetic interpolation. Among the steps described above, arithmetic interpolation of the density values at grid points requires arithmetic estimation of the density value in the position of a particular grid point from the density values of a plurality of pixels surrounding said grid point (i.e., the pixels within a region having a two-dimensional extent, with the grid point lying in the center) should be repeated for the two-dimensional distribution of the multiple grid points that are to be processed. Obviously, this involves quite complicated procedures.
As is clear from the above, the distortion correction involves the shift of the pixel positions represented by the yet to be corrected image data and, hence, the shape of the outer edge of the image represented by the as-corrected image data also changes from a rectangular to a non-rectangular form (such as a barrel or pincushion shape) as the result of the aforementioned aberration (distortion) correction. Consider, for example, the case of correcting an image which, due to distortion, has suffered from a xe2x80x9cpincushion-typexe2x80x9d geometric distortion as shown in FIG. 25A. After the correction, the figure of the outer edge of the image represented by the corrected image data is xe2x80x9cbarrel-shapedxe2x80x9d as shown in FIG. 25B. On the other hand, images generally have rectangular outer edges. Therefore, if the distortion corrected image data is simply output, blanks or regions having indeterminate density values occur in some areas of the output image (blanks or so-called xe2x80x9cpixel holesxe2x80x9d that appear in areas near the four corners of the image shown in FIG. 25B). The same defect occurs in the correction of chromatic aberration of magnification since it also involves the shift of pixel positions although it is very small.
As already mentioned, the content of image processing and the value of a parameter to be applied to a specific image processing scheme in the conventional image processing system have been set identical in specified directions, such as vertical and horizontal, of the image represented by image data. Hence, it has been impossible to perform different image processing schemes in different directions and this has reduced the latitude in the overall image processing operation.
Take, for example, the aforementioned case of performing electronic scaling with the conventional image processing system, in which the electronic scaling ratio is set to be identical in both the vertical and horizontal directions. In this situation, it has been impossible to perform a special image processing scheme such as a so-called xe2x80x9cfinish to slenderxe2x80x9d technique by which a human subject in the original image is finished to appear slender on the principal image.
The present invention has been accomplished under these circumstances and has as a first object providing an image processing method and apparatus that can output prints (photographs) of high quality reproducing high-quality images even if the original image has been taken with inexpensive cameras such as films with lens and compact cameras or taken with inexpensive digital cameras.
A second object of the invention is to provide a digital image processing method and apparatus which, by image processing, can correct the image aberrations due to poor lens performance (the deterioration in image quality due to lens aberrations) so that high-quality images free from distortion, color divergence and other defects can be output consistently without regard to the shooting magnification (focal length) even if the original image has been taken with inexpensive cameras such as films with lens and compact cameras or taken with inexpensive digital cameras.
A third object of the invention is to provide a digital image processing method and apparatus which, by image processing, can correct the image aberrations due to poor lens performance (the image deterioration due to lens aberrations) so that high-quality images free from distortion, color divergence and other defects can be output even if the original image has been taken with inexpensive cameras such as films with lens and compact cameras or taken with inexpensive digital cameras.
A fourth object of the invention is to provide an image processing method and apparatus which are capable of rapidly correcting aberrations such as distortion and chromatic aberration of magnification so that high-quality images free from distortion, color divergence and other defects can be produced at lower cost even if the original image has been taken with inexpensive cameras such as films with lens and compact cameras or taken with inexpensive digital cameras.
A fifth object of the invention is to provide an image processing method and apparatus that adopt a simple configuration and which can yet perform high-speed image processing on the image data with a higher degree of flexibility.
As already mentioned, films with lens and compact cameras do not justify the use of costly lenses, so they are unable to shoot images of sufficient quality to guarantee finished prints of high quality that reproduce high-quality images. To correct such image quality deteriorations due to poor lens performance, in particular, blurred images, the phase of the image has to be taken into account, namely, an inverse transform of PSF must be performed. However, this approach requires a bulky processing circuit and involves complicated processing procedures.
The present inventor made intensive studies in order to solve the aforementioned problems with the images taken with inexpensive cameras such as films with lens and found that by altering the intensity of sharpness enhancement processing (sharpening), particularly by performing more intense sharpening than in the usual case (in a default condition), prints of adequate image quality that were compensated for defocusing could be produced without difficult-to-perform PSF correction. The inventor also found that using an apparatus having a capability for sharpness enhancement, he could simply alter the intensity of the sharpening process to compensate for the deterioration in image quality due to poor lens performance without increasing the cost of finished products and so forth. The present invention has been accomplished on the basis of these findings.
In order to attain the first object described above, an image processing method of the present invention comprises the steps of: acquiring input image data from an image recorded optically with a lens for taking a picture; acquiring an information about the lens used to record the image; and performing image processing schemes including at least sharpness enhancement on the input image data using the acquired information about the lens to produce, output image data; wherein a type of the lens used is identified from the acquired information about the lens and an intensity of the sharpness enhancement of the corresponding image is altered in accordance with the identified type of the lens.
In this case, it is preferable that the intensity of the sharpness enhancement is altered uniformly for an entire part of the image in one frame or, alternatively, the image in the one frame is divided into a plurality of image areas and the intensity is altered for each of the divided image areas.
Preferably, the intensity of the sharpness enhancement is altered independently for each of three primary colors or altered uniformly for the three primary colors.
In the above-described image processing method, it is also preferable that a type of a film on which the image is recorded is further acquired and the intensity of the sharpness enhancement is altered in accordance with not only the type of the lens but also the acquired type of the film.
In the above-described image processing method, it is further preferable that lens characteristics of the taking lens are further acquired from the lens information and using the acquired lens characteristics and a position information for the recorded image, the sharpness, enhancement is performed on the input image data at different intensities on a pixel basis.
In the above-described image processing method, it is still further preferable that lens characteristics of the taking lens are further acquired from the lens information and using the acquired lens characteristics and a position information for the recorded image, the input image data for the image taken with the lens of the type requiring a change in the intensity of the sharpness enhancement is also subjected to aberration correction for correcting deterioration in image quality derived from the lens characteristics.
Preferably, the aberration correction is for correcting at least one of distortion, chromatic aberration of magnification, deterioration of marginal lumination, and defocusing derived from the lens characteristics.
Preferably, the defocusing is corrected by subjecting the sharpness enhancement to the input image data at different intensities on a pixel basis using the lens characteristics and the position information for the recorded image.
In the above-described image processing method, it is preferable that lens characteristics of the taking lens are further acquired from the lens information and using the acquired lens characteristics and a position information for the recorded image, at least one of aberration correction for correcting deterioration in image quality derived from the lens characteristics, electronic scaling and sharpness enhancement at different intensities on a pixel basis as the sharpness enhancement is performed on the input image data as one of the image processing schemes.
In the above-described image processing method of the first aspect, it is preferable that an information about the focal length effective at a time of recording the image is further acquired in addition to the lens information and lens characteristics of the taking lens are acquired from the lens information, and using the acquired lens characteristics, a position information for the recorded image and the information about the focal length, the input image data is subjected to aberration correction for correcting deterioration in image quality derived from the lens characteristics. Here, in this image processing method, it is also preferable that an information about a diaphragm used to record the image is further acquired and taking account into the obtained diaphragm information, correction of deterioration of marginal lumination is performed on the input image data as the aberration correction. Preferably, the lens characteristics of the taking lens are calculated in terms of the focal length effective at the time of recording the image using lens characteristics of the taking lens obtained previously at a plurality of focal lengths.
In the above-mentioned image processing of the first aspect, it is preferable that lens characteristics of the taking lens are further acquired from the lens information and using the acquired lens characteristics and a position information for the recorded image, the input image data is subjected to the image processing schemes in a first direction of the recorded image and a second direction crossing the first direction. Here, it is also preferable that the image processing schemes include at least one of correction of aberrations derived from the taking lens, electronic scaling and the sharpness enhancement. Preferably, the image processing schemes are performed in the first and second directions independently of each other. Preferably, an order of the image processing schemes is selectable in the first and second directions. In the above-described image processing method, it is further preferable that unidirectional image processing is further performed in at least one of the first and second directions. Preferably, contents of the image processing schemes are altered in each of the first and second directions. Preferably, alteration of the contents of the image processing schemes is change in values of parameters in the image processing schemes in each of the first and second directions.
Preferably, if the image processing schemes include at least correction of distortion and chromatic aberration of magnification, either an amount of correction of the chromatic aberration of magnification or an amount of correction of the distortion or both amounts differ between the first and second directions. Preferably, if the image processing schemes include at least correction of distortion and chromatic aberration of magnification the correction in whichever of the first and second directions that requires the chromatic aberration of magnification and the distortion to be corrected in smaller amounts precedes the correction in the other direction. Preferably, if the input image data is acquired by reading photoelectrically the recorded image with line sensors that have the first direction as a main scanning direction and the second direction as an auxiliary scanning direction, the image processing schemes in the second direction further, including correction of color divergence caused by the line sensors.
Moreover, in the above-described image processing method of the first aspect, if the image processing schemes include aberration correction for correcting aberrations in the image derived from the taking lens, preset parameters for correcting the aberrations which the taking lens used to record the image causes in a plane where the image is focused are scaled with at least one of an electronic scaling ratio for producing the output image data, number of input pixels in the input image data, a size of the input image and a size of the output image to produce aberration correction parameters that are related to the output image data on a pixel basis, and then is performed using the aberration correction to correct the aberrations in the image derived from the taking lens.
In order to attain the first object described above, an image processing apparatus of the first aspect of the present invention that acquires input image data from an image recorded optically with a lens for taking a picture and performs image processing schemes on the input image data to produce output image data, and which comprises first acquisition means for acquiring a lens information about the taking lens used to record the image; identifying means for identifying a lens type from the acquired lens information; and image processing means for performing at least sharpness enhancement of the image; wherein the image processing means alters an intensity of the sharpness enhancement of the corresponding image in accordance with an identification result of the lens type by the identifying means.
In this aspect, it is preferable that the image processing means comprises not only sharpness enhancing means for performing the sharpness enhancement on the image but also storage means for storing lens characteristics related to the lens type and aberration correcting means for receiving the lens characteristics of the corresponding lens type and correcting deterioration in image quality of the image based on a position information of the image and the lens characteristics, wherein correction the deterioration of the image quality by the aberration correcting means is also performed on the image that alters the intensity of the sharpness enhancement.
Preferably, the aberration correcting means corrects at least one of distortion, chromatic aberration of magnification and deterioration of marginal lumination derived from the lens characteristics.
Preferably, the image processing means performs the sharpness enhancement by the sharpness enhancing means after the correction of the image quality deterioration by the aberration correcting means.
Preferably, the aberration correcting means corrects the chromatic aberration of magnification and the distortion derived from the lens characteristics or further the deterioration of the marginal lumination derived from the lens characteristics, and wherein the aberration correcting means assigns one of three primary colors as a reference color, calculates offsets in image positions of the other colors from the image position of the reference color derived from the chromatic aberration of magnification, uses the offsets derived from the chromatic aberration of magnification and the offset in the image position of the reference color derived from the distortion to calculate appropriate positions of the respective images as corrected not only for the distortion but also for the chromatic aberration of magnification, and corrects the image quality deterioration based on the appropriate positions of the respective images or uses the appropriate positions of the respective images to correct the image quality deterioration and perform electronic scaling.
Preferably, the lens type that requires change in the intensity of the sharpness enhancement of the image is the lens of a film with lens.
Preferably, the image processing means alters:the intensity of the sharpness enhancement uniformly for an entire part of the image in one frame or divides the image in the one frame into a plurality of image areas and alters the intensity for each of the divided image areas.
Preferably, the image processing means alters the intensity of the sharpness enhancement independently for each color of three primary colors or uniformly for each color of the three primary colors.
Preferably, the image processing means further alters the intensity of the sharpness enhancement in accordance with a film type.
It is preferable that the above-described image processing apparatus of the first aspect further comprises: second acquisition means for acquiring an information about a focal length of the taking lens effective at the time of recording the image if the taking lens used to record the image is a lens of variable focal length; wherein the image processing means comprises storage means for storing lens characteristics of the taking lens used to record the image and aberration correcting means for acquiring the lens characteristics of the corresponding lens from the storage means in accordance with the lens information acquired by the first acquisition means and correcting aberrations in the image derived from the taking lens used to record the image using the acquired lens characteristics, the position information of the image and the information about the focal length of the taking lens acquired by the second acquisition means. In this case, it is also preferable that when the aberration correcting means corrects deterioration of marginal lumination, the second acquisition means further acquires an information about the diaphragm used at the time of image recording and the aberration correcting means additionally uses the diaphragm information to correct the deterioration of the marginal lumination. Preferably, the storage means stores, as the lens characteristics, the lens characteristics at a plurality of focal lengths of the lens and the aberration correcting means calculates the lens characteristics at the plurality of the focal lengths in terms of the focal length effective at the time of the image recording that was acquired by the second acquisition means to determine the lens characteristics of the lens at the focal length effective at the time of the image recording. Preferably, the aberration correcting means assigns one of three primary colors as a reference color, calculates offsets in image positions of the other colors from the image position of the reference color derived from the chromatic aberration of magnification, uses the offsets derived from the chromatic aberration of magnification and the offset in the image position of the reference color derived from the distortion to calculate appropriate positions of the respective images as corrected for both the distortion and the chromatic aberration of magnification, and corrects the distortion and the chromatic aberration of magnification based on the appropriate positions or uses the appropriate positions to perform electronic scaling.
It is further preferable that the above-described image processing apparatus of the first aspect further includes storage means for storing lens characteristics of the taking lens in accordance with the lens information about the taking lens and wherein the image processing means has bi-directional image processing means for performing the image processing schemes on the input image data in a first direction of the recorded image and a second direction crossing the first direction, using an information about pixel positions of an input image and the characteristics of the related taking lens as read from the storage means in accordance with the taking lens information acquired by the first acquisition means. Preferably, the bi-directional image processing means has at least one of a first and a second distortion correcting part that correct distortion in the first and second directions, respectively; a first and a second magnification chromatic aberration correcting part that correct chromatic aberration of magnification in the first and second directions, respectively; a first and a second marginal lumination deterioration correcting part that correct deterioration of marginal lumination in the first and second directions, respectively; a first and a second defocusing correcting part that correct defocusing in the first and second directions, respectively; a first and a second electronic scaling part that perform electronic scaling in the first and second directions, respectively; and a first and a second sharpening part that perform sharpness enhancement in the first and second directions, respectively. Preferably, the bi-directional image processing means has a first image processing part and a second image processing part that perform image processing schemes independently of each other in the first and second directions, respectively. Preferably, the bi-directional image processing means is capable of selecting an order of the image processing schemes that are performed by the first and second image processing parts. It is still further preferable that the above-described image processing apparatus further includes unidirectional image processing means for performing unidirectional image processing in at least one of the first and second directions. Preferably, if the input image data is acquired by line sensors that read the image recorded on the film, the unidirectional image processing means has a color divergence correcting part that corrects the color divergence derived from the line sensors.
It is preferable that the above-described image processing apparatus further includes control means for altering contents of the image processing schemes in each of the first and second directions. Preferably, the control means alters the contents of the image processing schemes by changing a kind or degree of the image processing schemes. Preferably, the control means alters the contents of the image processing schemes by changing values of parameters in the image processing schemes in each of the first and second directions. Preferably, the parameters in the image processing schemes are at least one of a filter coefficient of a filter used; a correction coefficient for correction of distortion; a correction coefficient for the correction of chromatic aberration of magnification; a correction coefficient for correction of deterioration of marginal lumination; a correction: coefficient for correction of defocusing; an electronic scaling ratio; and a coefficient of the sharpness enhancement.
Preferably, if the bi-directional image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, either an amount of correction of chromatic aberration of magnification or an amount of correction of distortion or both amounts in the distortion and magnification chromatic aberration correcting parts differ between the first and second directions. Preferably, if the bi-directional image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, correction in whichever of the first and second directions that requires the chromatic aberration of magnification and distortion to be corrected in smaller amounts precedes the correction in the other direction. It is preferable that if the input image data is acquired by photoelectric reading of the recorded image with line sensors that have the first direction as a main scanning direction and the second direction as an auxiliary scanning direction, the image processing apparatus described above further includes a color divergence correcting part that corrects the color divergence derived from the line sensors in the second direction.
It is preferable that the above-described image processing apparatus of the first aspect further includes storage means for storing parameters for correcting aberrations which the lens used to record the image causes in an imaging plane where the image is focused and the image processing means further includes: selection means for selecting a parameter which corrects the aberrations that the related taking lens causes on the imaging plane from the storage means in accordance with the lens information acquired by the first acquisition means; conversion means by which the parameter for correcting the aberrations on the imaging plane as selected by the selection means is scaled with at least one of an electronic scaling ratio for producing the output image data, number of input pixels in the input image data, size of an input image and the size of the output image, whereby the parameter is converted to an aberration correcting parameter that is related to the output image data on a pixel basis; and aberration correcting means which corrects the aberrations of the image derived from the image taking lens using the thus obtained, pixel-dependent aberration correcting parameter. Preferably, the image is one that is recorded on a photographic film and the size of the input image is equal to the size of the image as it is read from the photographic film.
In order to attain the second object described above, an image processing method of the second aspect of the resent invention that acquires not only input image data from an image recorded optically with a taking lens but also a lens information about the taking lens used to record the image and which performs image processing schemes on the input image data using the obtained lens information, thereby producing output image data, and which comprises the steps of: acquiring not only an information about focal length effective at the time of recording the image but also lens characteristics of the taking lens from the lens information; and correcting aberrations in the image derived from the taking lens used to record the image using the obtained lens characteristics, a position information for the recorded image and the information about the focal length.
Preferably, the aberrations comprise at least one of chromatic aberration of magnification, distortion, deterioration of marginal lumination and defocusing. It is preferable that the above-described image processing method further comprises steps of acquiring an information about a diaphragm used to record the image and correcting deterioration of marginal lumination of the input image data as the aberration taking account of the diaphragm information. Preferably, the characteristics of the taking lens are calculated in terms of the focal length effective at image recording using preliminarily obtained lens characteristics of the taking lens at a plurality of focal lengths. Preferably, the correction of aberrations includes distortion and chromatic aberration of magnification and comprises the steps of assigning one of three primary colors as a reference color, calculating offsets in image positions of the other colors from the image position of the reference color derived from the chromatic aberration of magnification, calculating appropriate positions of the respective images as corrected for both the distortion and the chromatic aberration of magnification by using the offsets derived from the chromatic aberration of magnification and the offset in the image position of the reference color derived from the distortion, and performing either the correction of the distortion and the chromatic aberration of magnification or electronic scaling or both by using the appropriate positions.
In order to the second object described above, an image processing apparatus of the second aspect of the present invention which acquires input image data from the image recorded optically with a taking lens of variable focal length and performs image processing schemes on the input image data to produce output image data, and which comprises: first acquisition means for acquiring an information about the taking lens used to record the image, second acquisition means for acquiring an information about the focal length of the taking lens effective at the time of recording the image, storage means for storing lens characteristics of the lens used to record the image, and aberration correcting means which, in accordance with the lens information acquired by the first acquisition means, obtains the lens characteristics of the corresponding lens from the storage means and which uses the obtained lens characteristics, a position information for the image and the information about the focal length of the lens acquired by the second acquisition means, thereby correcting the aberrations in the image derived from the taking lens used to record the image.
In this case, it is preferable that the aberrations comprise at least one of chromatic aberration of magnification, distortion, deterioration of marginal lumination and defocusing.
Preferably, if the aberration correcting means is to correct deterioration of marginal lumination, the second acquisition means also acquires an information about a diaphragm used to record the image and the aberration correcting means corrects the marginal lumination deterioration taking account of the diaphragm information.
Preferably, the storage means stores, as the lens characteristics, characteristics of the lens at a plurality of focal lengths and the aberration correcting means calculates the lens characteristics at the plurality of focal lengths in terms of the focal length effective at the time of recording the image that was acquired by the second acquisition means, thereby determining the lens characteristics of the lens at the focal length effective at the time of image recording.
Preferably, the aberration correcting means assigns one of three primary colors as a reference color, calculates offsets in image positions of the other colors from the image positions of the reference color derived from chromatic aberration of magnification, calculates the appropriate positions of the respective images as corrected for both distortion and chromatic aberration of magnification by using the offsets derived from chromatic aberration of magnification and the offset in the image position of the reference color derived from distortion, and corrects the distortion and chromatic aberration of magnification based on the appropriate positions or performs electronic scaling by using the appropriate positions.
In order to attain the third object described above, an image processing method of the third aspect of the present invention which acquires input image data from an optically recorded image and performs image processing schemes on the input image data to produce output image data, and which comprises the steps of: scaling preset parameters for correcting the aberrations which the lens used to record the image causes in the plane where the image is focused by means of at least one of an electronic scaling ratio for producing the output image data, number of input pixels in the input image data, a size of the input image and a size of the output image to produce aberration correction parameters that are related to the output image data on a pixel basis; and correcting the aberrations in the image derived from the lens used to record the image by using the aberration correction parameters on the pixel basis.
In order to attain the third object described above, an image processing apparatus of the third aspect of the present invention which acquires input image data from an optically recorded image and performs image processing schemes on the input image data to produce output image data, and which comprises: acquisition means for acquiring an information about taking lens used to record the image; storage means for storing parameters for correcting the aberrations which the lens used to record the image causes in an imaging plane where the image is focused; selection means by which a parameter for correcting the aberrations which the related lens causes on the imaging plane is selected from the storage means in accordance with the lens information acquired by the first acquisition means; conversion means by which the parameter for correcting the aberrations on the imaging plane as selected by the selection means is scaled with at least one of an electronic scaling ratio for producing the output image data, number of input pixels in the input image data, a size of the input image and a size of the output image, whereby the parameter is converted to an aberration correcting parameter that is related to the output image data on a pixel basis; and aberration correcting means which corrects the image aberrations derived from the image taking lens by using the aberration correcting parameter on the pixel basis converted by the conversion means .
In this aspect, it is preferable that the image is one that is recorded on a photographic film and the size of the input image i s equal to a size of the image as it is read from the photographic film. Preferably, the aberration comprises at least one of chromatic aberration of magnification, distortion, deterioration of marginal lumination and defocusing.
In order to attain the fourth object described above, an image processing method of the fourth aspect of the present invention comprises the steps of: acquiring not only input image data from the image recorded optically with a taking lens but also an information about the taking lens used to record the image; obtaining lens characteristics of the taking lens from the acquired lens information; and performing image processing schemes on the input image data by using the obtained lens characteristics and a position information for the recorded image; wherein the input image data is subjected to the image processing schemes a first direction of the recorded image and a second direction crossing the first direction. Preferably, the image processing schemes include at least one of correction of aberrations derived from the taking lens, electronic scaling and sharpening. Preferably, the correction of the aberrations includes at least one of distortion, chromatic aberration of magnification, deterioration of marginal lumination, and defocusing. Preferably, the image processing schemes are performed in the first and second directions independently of each other. Preferably, an order of the image processing schemes is selectable in the first and second directions. It is preferable that the above-described image processing method further comprises the step of performing unidirectional image processing in at least one of the first and second directions. Preferably, when the input image data is acquired by line sensors that read the image recorded on a film, the unidirectional image processing is color divergence correction that corrects the color divergence derived from the line sensors. Preferably, contents of the image processing schemes are altered in each of the first and second directions. Preferably, the contents of the image processing schemes are altered by changing a kind or degree of the image processing schemes. Preferably, the contents of the image processing schemes are altered by changing values of parameters in the image processing schemes in each of the first and second directions. Preferably, parameters in the image processing schemes are at least one of a filter coefficient of a filter used; a correction coefficient for correction of distortion; a correction coefficient for correction of chromatic aberration of magnification; a correction coefficient for correction of deterioration of marginal lumination; a correction coefficient for correction of defocusing; an electronic scaling ratio; and a coefficient of sharpening.
Preferably, if the image processing schemes include at least correction of distortion and chromatic aberration of magnification, either an amount of correction of chromatic aberration of magnification or an amount of correction of distortion or both amounts differ between the first and second directions. Preferably, if the image processing schemes include at least correction of distortion and chromatic aberration of magnification, the correction in whichever of the first and second directions that requires the chromatic aberration of magnification and distortion to be corrected in smaller amounts precedes the correction in the other direction. Preferably, if the input image data is acquired by photoelectric reading of the recorded image with line sensors that have the first direction as a main scanning direction and the second direction as an auxiliary scanning direction, the image processing schemes in the second direction further include correction of color divergence caused by the line sensors. Preferably, the first and second directions cross at right angles.
In order to attain the fourth object described above, an image processing apparatus of the fourth aspect of the present invention which acquires input image data from the image recorded optically with a taking lens and performs specified image processing schemes on the input image data to produce output image data, and which comprises: acquisition means for acquiring an information about the taking lens used to record the image; storage means for storing characteristics of the taking lens in accordance with the information about the taking lens; and image processing means which performs the image processing schemes on the input image data in both a first direction of the recorded image and a second direction crossing the first direction, by using an information about pixel positions of the input image and the characteristics of the related taking lens as read from the storage means in accordance with the information about the taking lens acquired by the acquisition means.
In this aspect, it is preferably that the image processing means has at least two parts of first and second distortion correcting parts that correct distortion in the first and second directions, respectively; first and second magnification chromatic aberration correcting parts that correct chromatic aberration of magnification in the first and second directions, respectively; first and second marginal lumination deterioration correcting parts that correct deterioration of marginal lumination in the first and second directions, respectively; first and second defocusing correcting parts that correct defocusing in the first and second directions, respectively; first and second electronic scaling parts that perform electronic scaling in the first and second directions, respectively; and first and second sharpening parts that perform sharpening in the first and second directions, respectively. Preferably, the image processing means has a first image processing part and a second image processing part that perform image processing schemes independently of each other in the first and second directions. Preferably, the image processing means is capable of selecting an order of the image processing schemes that are performed by the first, and second image processing parts. It is preferable that the above-described image processing apparatus further includes unidirectional image processing means for performing unidirectional image processing in at least one of the first and second directions. Preferably, if the input image data is acquired by line sensors that read the image recorded on a film, the unidirectional image processing means has a color divergence correcting part that corrects the color divergence derived from the line sensors.
It is also preferable that the above-described image processing apparatus further includes control means for altering contents of the image processing schemes in each of the first and second directions. Preferably, the control means alters the contents of the image processing schemes by changing a kind or degree of the image processing schemes. Preferably, the control means alters the contents of the image processing schemes by changing values of parameters in the image processing schemes in each of the first and second directions. Preferably, the parameters in the image processing schemes are at least one of a filter coefficient of a filter used; a correction coefficient for correction of distortion; a correction coefficient for correction of chromatic aberration of magnification; a correction coefficient for correction of deterioration of marginal lumination; a correction coefficient for correction of defocusing; an electronic scaling ratio; and a coefficient of sharpening.
Preferably, if the image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, either an amount of correction of chromatic aberration of magnification or an amount of correction of distortion or both amounts in the distortion and magnification chromatic aberration correcting parts differ between the first and second directions. Preferably, if the image processing means includes at least a distortion correcting part and a magnification chromatic aberration correcting part, correction in whichever of the first and second directions that requires chromatic aberration of magnification and distortion to be corrected in smaller amounts precedes the correction in the other direction. Preferably, if the input image data is acquired by photoelectric reading of the recorded image with line sensors that have the first direction as a main scanning direction and the second direction as an auxiliary scanning direction, it further includes a color divergence correcting part that corrects the color divergence derived from the line sensors in the second direction. Preferably, the image processing means allows the first and second directions to cross at right angles.
In order to attain the fifth object described above, an image processing method of the fifth aspect of the present invention comprises the step of subjecting image data representing an image recorded on an image recording medium to image processing schemes in either a first direction or a second direction crossing the first, direction or both directions, wherein if the image processing schemes are to be performed in both the first and second directions, contents of the image processing schemes in each of the first and second directions are altered. In this aspect, the alternation of contents of the image processing schemes in each of the first and second directions includes cases that the kind of the image processing scheme is altered and that the degree of the image processing scheme is altered while the kind of the image processing scheme is the same.
For example, if the above-mentioned image data is obtained with the three-line color CCD corresponding to each of R, G and B components, the image which is formed on the basis of the above-mentioned image data may have color divergence in the direction (auxiliary scanning direction) that is perpendicular to the direction (main scanning direction) in which lines of the three-line color CCD extend. In this case, the image processing may be performed to correct the color divergence only in the auxiliary direction. Therefore, the above-mentioned color divergence can be corrected by the image processing only in one direction which applies the auxiliary direction as either one of the first and second directions in the image processing method of the present aspect, as well as the above-mentioned correction processing of color divergence as the image processing whereby the image processing only in one direction can be performed at a higher speed than the image processing in both first and second directions can.
Moreover, for example, when the image recorded in the image recording medium is projected through the lens so that the correction of the aberrations of the above lens is performed as the image processing, the correction processing must normally be performed in both main,scanning and auxiliary scanning directions; hence, by adopting the main scanning and the auxiliary scanning directions as the first and second directions respectively in the image processing method of the present aspect, the correction of the aberrations of the lens is performed in both directions as the image processing in this method. In this case, in the method of the present aspect, aberration correction is performed in each direction of the first and second directions. Therefore, a moving direction of a pixel position caused by the aberration correction is limited to one direction so that, for example, the interpolation operation can be performed using data of pixels lined along one direction to allow the aberration correction processing to be performed in a high speed.
On the other hand, for example, if the image recorded in the image recording medium is projected through the lens, as well as the data thereof are obtained corresponding to each of R, G and B components with the three-line color CCD, it may be in cases required that only aberration correction of lens is performed in the main scanning direction while both the correction processing of color divergence caused by the three-line color CCD and the aberration correction processing of lens are performed. In this case, the aberration correction of lens in the auxiliary direction is normally performed with a different correction coefficient from that of the aberration correction in the main scanning direction. Also in this case, image processing in each direction can be realized by adopting the main scanning and auxiliary scanning directions as the first and second directions in he method of the present aspect respectively, as well as the aberration correction as the image processing for the first direction, and the above-mentioned color divergence correction and the aberration correction which has t different correction coefficient from that for the first direction as the image processing for the second direction.
Thus, according to the method of the present aspect, the image data representing the image recorded on the image recording medium is subjected to the image processing schemes in either the first direction or the second direction crossing said first direction or both directions, so that the image processing of the image data can be performed at high speed. As well as, if the image processing is to be performed in both the first and second directions, the contents of the image processing in the first direction are changed from those in the second direction; hence, compared to the case of making no changes in their contents, the intended image processing schemes can be performed with a higher degree of flexibility.
In the present aspect of the invention, a preferred embodiment of the image processing method is characterized in that the contents of the image processing schemes are altered by changing values of parameters in the image processing schemes in each of the first and second directions. Therefore, for example, if the electronic scaling processing is performed as the image processing schemes, the parameter in the image processing schemes in each of the first and second directions is rendered to the electronic scaling ratio and thereby the aspect ratio of the image represented by the image data after the image processing allow to differ from the aspect ratio of the image represented by the image data before the image processing, for example, it is able to perform a finish to slender.
Thus, according to the preferred embodiment, the contents of image processing schemes are altered by changing the values of parameters in the image processing schemes in each of the first and second directions; hence, highly flexible image processing can be accomplished by a simple method of only changing the values of pertinent parameters.
In order to attain the fifth object described above, an image processing apparatus of the fifth aspect of the present invention comprises: image processing means for performing unidirectional image processing schemes on image data that represents an image recorded on an image recording medium; and control means for controlling the image processing means such that image processing schemes are performed on the image data in either a first direction or a second direction crossing the first direction or both directions, wherein if the control means controls the image processing means such that the image processing schemes are performed in both the first and second directions, the control means alters contents of the image processing schemes in each of the first and second directions.
According to the image processing apparatus of the present aspect, the image processing means for performing the unidirectional image processing schemes on the image data that represents the image recorded on the image recording medium is provided, and the image processing means is controlled by the control means such that the image processing schemes are performed on the image data in at least one of the first direction and the second direction crossing the first direction, as well as, if image processing means is controlled such that the image processing schemes are performed in both the first and second directions, contents of the image processing schemes in each of the first and second directions are altered.
Thus, according to the apparatus of the present aspect, in the same way as the method of the present aspect, since the image data representing the image recorded on the image recording medium is subjected to image processing in either the first direction or the second direction crossing said first direction or both directions, the image processing of the image data can be performed at high speed. And if the image processing is to be performed in both the first and second directions, the contents of the image processing in the first direction are changed from those in the second direction; hence, compared to the case of making no changes in their contents, the intended image processing schemes can be performed with a higher degree of flexibility. As well as, if the image processing is to be performed in both the first and second directions, the image processing in the first direction and that in the second direction can be executed by a single image processing means and this helps simplify the construction of the image processing apparatus.
A preferred embodiment of the image processing apparatus is characterized in that the control means alters the contents of image processing schemes by changing the values of parameters in image processing schemes in each of the first and second directions. According to the preferred embodiment, the values of the parameters of the image processing schemes in each of the first and second directions are varied by the control means in the apparatus of the present aspect, thereby the contents of the image processing schemes are changed. Thus, according to the apparatus of the preferred embodiment, since the contents of the image processing schemes are altered by changing the values of the parameters in the image processing schemes in each of the first and second directions, in the same way as the preferred embodiment of the method, highly flexible image processing can be accomplished by a simple method of only changing the values of pertinent parameters. It should be noted that as the parameters in the image processing schemes described above, at least one of a filter coefficient of a filter used, a coefficient of sharpening, a correction coefficient for correction of aberrations, and an electronic scaling ratio can be applied.